Constrained Second - Order RecurrentNetworks for Finite - State

This paper presents an improved training algorithm for second-order dy-namical recurrent networks applied to the problem of nite-state au-tomata (FSA) induction. Second-order networks allow for a natural encoding of nite-state automata in which each second-order connection weight corresponds to one transition in a nite-state automaton. In practice , however, when trained using gradient descent, these networks seldom assume this type of encoding and sophisticated algorithms must be used to extract the encoded automata. This paper suggests a simple modiica-tion to the standard error function for second-order dynamical recurrent networks which encourages these networks to assume natural FSA en-codings when trained using gradient descent. This obviates the need for cluster-based extraction techniques and provides a simple method for guaranteeing the stability of the network for arbitrarily long sequences. Initial results also suggest that fewer training strings must be presented to achieve convergence using the modiied error.